Network Architecture Research Articles

Potential of Large Language Models in Health Care: Delphi Study.

A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications. The aim of this adapted Delphi study was to collect researchers' opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care. We invited researchers in the fields of health informatics, nursing informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items. The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data. Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.

Resilient data-driven non-intrusive load monitoring for efficient energy management using machine learning techniques

The integration of smart homes into smart grids presents numerous challenges, particularly in managing energy consumption efficiently. Non-intrusive load management (NILM) has emerged as a viable solution for optimizing energy usage. However, as smart grids incorporate more distributed energy resources, the complexity of demand-side management and energy optimization escalates. Various techniques have been proposed to address these challenges, but the evolving grid necessitates intelligent optimization strategies. This article explores the potential of data-driven NILM (DNILM) by leveraging multiple machine learning algorithms and neural network architectures for appliance state monitoring and predicting future energy consumption. It underscores the significance of intelligent optimization techniques in enhancing prediction accuracy. The article compares several data-driven mechanisms, including decision trees, sequence-to-point models, denoising autoencoders, recurrent neural networks, long short-term memory, and gated recurrent unit models. Furthermore, the article categorizes different forms of NILM and discusses the impact of calibration and load division. A detailed comparative analysis is conducted using evaluation metrics such as root-mean-square error, mean absolute error, and accuracy for each method. The proposed DNILM approach is implemented using Python 3.10.5 on the REDD dataset, demonstrating its effectiveness in addressing the complexities of energy optimization in smart grid environments.

Spintronic Artificial Synapses Using Voltage‐Controlled Multilevel Magnetic States

AbstractNeuromorphic computing offers energy‐efficient computations for large‐scale data processing compared to conventional von Neumann computing. The artificial synapse, a key element for learning and memory operations in neuromorphic computing, requires multi‐state characteristics and the capability to change and store its states. The implementation of hardware‐based artificial synapses using nonvolatile memory provides significant advantages in terms of energy consumption and circuit area compared to their currently employed CMOS‐based counterparts. In this regard, spintronic devices have emerged as a promising candidate due to their desirable properties for artificial synapses, including multilevel formability, non‐volatility, and outstanding writing performance. In this study, spintronic artificial synapses utilizing voltage‐controlled multilevel magnetic states and energy‐ and area‐efficient artificial neural network architectures associated with them are demonstrated. The multilevel states are created by gradually modulating the magnetic easy‐axis orientation from perpendicular to in‐plane and vice versa, which is achieved either by sequentially applying gate voltage pulses or by adjusting the pulse width of the gate voltage. Based on these spintronic artificial synapses, convolutional neural network (CNN) and spiking neural network (SNN) architectures are constructed, demonstrating high recognition accuracy for the MNIST dataset with improved energy efficiency and a reduced circuit area.

UAV and Deep Learning: Detection of selected riparian species along the Ganga River

Abstract. Environmental protection and sustainable natural resource management are being recognised worldwide as essential goals to safe guard human health and well-being. Riparian zones, that face the highest decline in freshwater biodiversity, are of prime conservation priority because they are essential for regulating climate, preserving aquatic-terrestrial biodiversity, maintaining ground water recharge and restoring rivers. In today's fast-paced data-driven environment, artificial intelligence (AI) is the precise answer to a wide range of problems including biodiversity conservation and wildlife management. Leveraging advancements like Uncrewed/Unmanned Aerial Vehicles (UAVs) and AI has resulted innovative strides in wildlife conservation. This study utilised UAV imagery to record high-resolution data of aquatic habitat and species along the Ganga River and employed deep learning algorithms to analyse the data. Through extensive field surveys in the Hastinapur Wildlife Sanctuary, 7,025 photos representing a variety of environments, including 20,000 annotated samples of aquatic animals such as turtles and gharials were generated. Vision based computing capabilities such as pattern recognition model were developed to identify these species. To enrich and enhance the dataset and model, we used different image pre-processing techniques. Slight rotation (±5 degrees), minor cropping (up to 10%), and adjustments in brightness, saturation, and shear (±15%) were applied. Controlled blur (up to 0.5%) and exposure modifications (±5%) were also implemented on the image dataset to improve accuracy. Three Convolutional Neural Network (CNN) architectures, single-stage detectors named YOLO v7, YOLO v8, and Roboflow 3.0, were used for detecting the select species. Results show YOLO v8 excels, achieving mean average precision (mAP) of 98.8% for gharial and 92.2% for turtle detection, with a rapid average detection time of 0.308 seconds per frame at 3200 × 3200 resolution. Additionally, our model demonstrates real-time species detection capability through innovative frame sampling techniques with UAVs. This methodology provides promising technique to collect scientific data on IUCN red listed turtles and critically endangered gharials, allowing detection, monitoring, and real time counting with minimal intrusion. In conclusion, the fusion of UAVs and deep learning promises to revolutionize habitat monitoring, aiding conservation decision-making.

BACK-to-MOVE: Machine learning and computer vision model automating clinical classification of non-specific low back pain for personalised management.

Low back pain (LBP) is a major global disability contributor with profound health and socio-economic implications. The predominant form is non-specific LBP (NSLBP), lacking treatable pathology. Active physical interventions tailored to individual needs and capabilities are crucial for its management. However, the intricate nature of NSLBP and complexity of clinical classification systems necessitating extensive clinical training, hinder customised treatment access. Recent advancements in machine learning and computer vision demonstrate promise in characterising NSLBP altered movement patters through wearable sensors and optical motion capture. This study aimed to develop and evaluate a machine learning model (i.e., 'BACK-to-MOVE') for NSLBP classification trained with expert clinical classification, spinal motion data from a standard video alongside patient-reported outcome measures (PROMs). Synchronised video and three-dimensional (3D) motion data was collected during forward spinal flexion from 83 NSLBP patients. Two physiotherapists independently classified them as motor control impairment (MCI) or movement impairment (MI), with conflicts resolved by a third expert. The Convolutional Neural Networks (CNNs) architecture, HigherHRNet, was chosen for effective pose estimation from video data. The model was validated against 3D motion data (subset of 62) and trained on the freely available MS-COCO dataset for feature extraction. The Back-to-Move classifier underwent fine-tuning through feed-forward neural networks using labelled examples from the training dataset. Evaluation utilised 5-fold cross-validation to assess accuracy, specificity, sensitivity, and F1 measure. Pose estimation's Mean Square Error of 0.35 degrees against 3D motion data demonstrated strong criterion validity. Back-to-Move proficiently differentiated MI and MCI classes, yielding 93.98% accuracy, 96.49% sensitivity (MI detection), 88.46% specificity (MCI detection), and an F1 measure of .957. Incorporating PROMs curtailed classifier performance (accuracy: 68.67%, sensitivity: 91.23%, specificity: 18.52%, F1: .800). This study is the first to demonstrate automated clinical classification of NSLBP using computer vision and machine learning with standard video data, achieving accuracy comparable to expert consensus. Automated classification of NSLBP based on altered movement patters video-recorded during routine clinical examination could expedite personalised NSLBP rehabilitation management, circumventing existing healthcare constraints. This advancement holds significant promise for patients and healthcare services alike.

Minimum Congestion and Wirelength of Embedding the nth Cartesian Product of K4 into Various Kinds of Grid Networks

Embedding an interconnection network into another network is one of the main problems in parallel processing and computing systems. Interconnection networks in multiprocessing systems facilitate effective communication among various system components. Obtaining the minimum cutwidth, congestion, and wirelength in graph embedding problems is of great significance for network design and architecture, where the minimum is taken over all embedding of guest graphs into host graphs, and addressing these issues can reduce time and cost in embedded design. This work aims to accurately calculate the minimum congestion and wirelength for the [Formula: see text] (the [Formula: see text]th Cartesian product of [Formula: see text]) layout into [Formula: see text]-dimensional, [Formula: see text]-dimensional, and [Formula: see text]-dimensional grids by the optimal solution of edge isoperimetric problem on [Formula: see text].

An Ensemble Learning Framework for Anomaly Detection of Important Geographical Entities

Abstract. Due to the complex landforms and the limited resolution of remote sensing imagery, it is difficult to avoid the problem of incorrectly capturing geographical entities, such as buildings. Therefore, anomaly detection of important geographical entities is of great significance to ensure the authenticity and accuracy of geographical entity data. In this paper, we propose an ensemble learning framework for anomaly detection of geographical entity by aggregating the predicted labels generated by multiple deep learning models. In detail, we explore multiple change detection and semantic segmentation model and fully utilize the advantages of various deep learning neural network architectures. The proposed anomaly detection strategy of buildings has been performed on two benchmark datasets, including WHU Building change detection dataset and LEVIR building change detection dataset, the experimental results prove that the proposed method can achieve a more robust and better performance than using single change detection model in terms of quantitative performance and visual performance.

Underwater Multi-Channel MAC with Cognitive Acoustics for Distributed Underwater Acoustic Networks

The advancement of underwater cognitive acoustic network (UCAN) technology aims to improve spectral efficiency and ensure coexistence with the underwater ecosystem. As the demand for short-term underwater applications operated under distributed topologies, like autonomous underwater vehicle cluster operations, continues to grow, this paper presents Underwater Multi-channel Medium Access Control with Cognitive Acoustics (UMMAC-CA) as a suitable channel access protocol for distributed UCANs. UMMAC-CA operates on a per-frame basis, similar to the Multi-channel Medium Access Control with Cognitive Radios (MMAC-CR) designed for distributed cognitive radio networks, but with notable differences. It employs a pre-determined data transmission matrix to allow all nodes to access the channel without contention, thus reducing the channel access overhead. In addition, to mitigate the communication failures caused by randomly occurring interferers, UMMAC-CA allocates at least 50% of frame time for interferer sensing. This is possible because of the fixed data transmission scheduling, which allows other nodes to sense for interferers simultaneously while a specific node is transmitting data. Simulation results demonstrate that UMMAC-CA outperforms MMAC-CR across various metrics, including those of the sensing time rate, controlling time rate, and throughput. In addition, except for in the case where the data transmission time coefficient equals 1, the message overhead performance of UMMAC-CA is also superior to that of MMAC-CR. These results underscore the suitability of UMMAC-CA for use in challenging underwater applications requiring multi-channel cognitive communication within a distributed network architecture.

Unifying O(3) Equivariant Neural Networks Design with Tensor-Network Formalism

Abstract Many learning tasks, including learning potential energy surfaces from ab initio calculations, involve global spatial symmetries and permutational symmetry between atoms or general particles. Equivariant graph neural networks are a standard approach to such problems, with one of the most successful methods employing tensor products between various tensors that transform under the spatial group. However, as the number of different tensors and the complexity of relationships between them increase, maintaining parsimony and equivariance becomes increasingly challenging. In this paper, we propose using fusion diagrams, a technique widely employed in simulating SU($2$)-symmetric quantum many-body problems, to design new equivariant components for equivariant neural networks. This results in a diagrammatic approach to constructing novel neural network architectures. When applied to particles within a given local neighborhood, the resulting components, which we term "fusion blocks," serve as universal approximators of any continuous equivariant function defined on the neighborhood. We incorporate a fusion block into pre-existing equivariant architectures (Cormorant and MACE), leading to improved performance with fewer parameters on a range of challenging chemical problems. Furthermore, we apply group-equivariant neural networks to study non-adiabatic molecular dynamics of stilbene cis-trans isomerization and water adsorption cycles in porous materials, such as metal-organic frameworks. Our approach, which combines tensor networks with equivariant neural networks, suggests a potentially fruitful direction for designing more expressive equivariant neural networks.

Trustworthy decentralization based on blockchain tools for social network architectures

Trustworthy decentralization based on blockchain tools for social network architectures

Automatic Image Captioning using Deep Learning

Image captioning aims to detect this information by describing the image content through image and text processing techniques. Now-a-days image captioning is one of the recent and growing research problem. Day by day various solutions are being introduced for solving the problem. Even though, many solutions are already available, a lot of attention is still required for getting better and precise results. So, we came up with the idea of developing a image captioning model using different combinations of Convolutional Neural Network architecture along with Long Short Term Memory to get better results. We have used three combination of CNN and LSTM for developing the model. The proposed model is trained with three Convolutional Neural Network architecture such as Inception-v3,Xception, ResNet50 for feature extraction and Long ShortTerm Memory for generating the relevant captions. Among these models, the best combination based on the accuracy of the model. It is trained using the Flicker8k dataset

TEDNet: Twin Encoder Decoder Neural Network for 2D Camera and LiDAR Road Detection

Abstract Robust road surface estimation is required for autonomous ground vehicles to navigate safely. Despite it becoming one of the main targets for autonomous mobility researchers in recent years, it is still an open problem in which cameras and LiDAR sensors have demonstrated to be adequate to predict the position, size and shape of the road a vehicle is driving on in different environments. In this work, a novel Convolutional Neural Network model is proposed for the accurate estimation of the roadway surface. Furthermore, an ablation study has been conducted to investigate how different encoding strategies affect model performance, testing 6 slightly different neural network architectures. Our model is based on the use of a Twin Encoder–Decoder Neural Network (TEDNet) for independent camera and LiDAR feature extraction and has been trained and evaluated on the Kitti–Road dataset. Bird’s Eye View projections of the camera and LiDAR data are used in this model to perform semantic segmentation on whether each pixel belongs to the road surface. The proposed method performs among other state-of-the-art methods and operates at the same frame rate as the LiDAR and cameras, so it is adequate for its use in real-time applications.

Phenotyping methodologies of log end splitting in eucalyptus (Eucalyptus spp.)

AbstractThis study addresses the crucial consideration of log end splitting in breeding programmes for treated wood. There is a paucity of research focused on efficiently optimizing the phenotyping process for this particular trait. The study aimed to compare methodologies for log end splitting phenotyping and develop an image‐based crack evaluation approach. Initially, 32 eucalyptus clones underwent phenotyping using manual measurement, digital image analysis and visual evaluation. Results showed similar phenotypic values, but image analysis demonstrated better clone discrimination, reducing evaluation time to 78 h compared to manual measurement. The second part focused on testing convolutional neural network architectures (UNet, LinkNet and FPN) using real and synthetic images. U‐Net exhibited slight superiority based on higher Intersection over Union (IoU) values, exhibiting a high correlation (.89) with true values. This approach significantly reduced evaluation time to approximately 10.15 h, emphasizing its efficiency compared to traditional methods.

Evaluating the performance of metaheuristic-tuned weight agnostic neural networks for crop yield prediction

AbstractThis study explores crop yield forecasting through weight agnostic neural networks (WANN) optimized by a modified metaheuristic. WANNs offer the potential for lighter networks with shared weights, utilizing a two-layer cooperative framework to optimize network architecture and shared weights. The proposed metaheuristic is tested on real-world crop datasets and benchmarked against state-of-the-art algorithms using standard regression metrics. While not claiming WANN as the definitive solution, the model demonstrates significant potential in crop forecasting with lightweight architectures. The optimized WANN models achieve a mean absolute error (MAE) of 0.017698 and an R-squared ($$R^2$$ R 2 ) score of 0.886555, indicating promising forecasting performance. Statistical analysis and Simulator for Autonomy and Generality Evaluation (SAGE) validate the improvement significance and feature importance of the proposed approach.

An efficient CNN-transformer hybrid approach for water turbine unit failure prediction

Water Turbine Unit is the core equipment of water power generation. It is the most important research object in hydroelectric power. The analysis of daily monitoring data of Water Turbines can evaluate its running state and avoid the loss caused by failure. In this paper, A hybrid neural network architecture which CNN combines Transformer is proposed to evaluate the health state of water turbines based on multivariate long time series data. It has two core components: (i) dilated convolution applies to capture low-level and local semantic information, then the output of convolutional layers is divided into subseries-level patches by time, these patches are regarded as input tokens of Transformer layers; (ii) utilizing self-attention of Transformer to extract high-level and global semantic information. Patching design naturally has benefit as follow: (i) local semantic information is retained in Transformer input tokens and high-level semantic confirm to common sense of human understanding through low-level construction; (ii) the length of Transformer input sequence is greatly shorted and attention can be more concentrated compared with point-wise form. Meanwhile, the computation and memory usage reduces at the same time. The experimental result indicates that the hybrid architecture can achieves excellent performance in time series understanding.

An efficient deep neural network for automatic classification of acute intracranial hemorrhages in brain CT scans

BackgroundRecent advancements in deep learning models have demonstrated their potential in the field of medical imaging, achieving remarkable performance surpassing human capabilities in tasks such as classification and segmentation. However, these modern state-of-the-art network architectures often demand substantial computational resources, which limits their practical application in resource-constrained settings. This study aims to propose an efficient diagnostic deep learning model specifically designed for the classification of intracranial hemorrhage in brain CT scans. MethodOur proposed model utilizes a combination of depthwise separable convolutions and a multi-receptive field mechanism to achieve a trade-off between performance and computational efficiency. The model was trained on RSNA datasets and validated on CQ500 dataset and PhysioNet dataset. ResultThrough a comprehensive comparison with state-of-the-art models, our model achieves an average AUROC score of 0.952 on RSNA datasets and exhibits robust generalization capabilities, comparable to SE-ResNeXt, across other open datasets. Furthermore, the parameter count of our model is just 3 % of that of MobileNet V3. ConclusionThis study presents a diagnostic deep-learning model that is optimized for classifying intracranial hemorrhages in brain CT scans. The efficient characteristics make our proposed model highly promising for broader applications in medical settings.

Competition of systematically optimized deep neural networks for the estimation of tensile behavior of aluminum-magnesium alloy

Aluminum-magnesium (Al-Mg) alloys are prevalently employed within the aerospace sector. This research engaged a suite of deep learning approaches, encompassing the Artificial Neural Network (ANN), Gated Recurrent Unit (GRU) networks, Long-Short Term Memory (LSTM), and simple Recurrent Neural Network (RNN) to evaluate their predictive efficacy regarding the tensile strength and stiffness of Al-Mg alloys obtained from molecular dynamics simulation. The Taguchi method was initially applied to refine the architecture of each deep neural network (DNN), followed by a comparative analysis of their optimized configurations. The findings of this investigation revealed that the refined simple RNN and LSTM models exhibited superior predictive accuracy for estimating the strength and stiffness of the alloy, respectively. Moreover, the study elucidated that DNNs equipped with memory capabilities outstripped traditional ANNs in forecasting the tensile properties of Al-Mg alloys.

The application of artificial neural networks to forecast financial time series

Abstract The amount of information that is produced on a daily basis in the financial markets is vast and complex; consequently, the development of systems that simplify decision-making is an essential endeavor. In this article, several intelligent systems are proposed and tested to predict the closing price of the IBEX 35 index using more than ten years of historical data and five distinct architectures for neural networks. A multi-layer perceptron was the first step, followed by a simple recurrent neural network, a gated recurrent unit network and two long-short-term memory (LSTM) networks. The results of the analyses performed on these models have demonstrated a powerful capacity for prediction. Additionally, the findings of this research point to the fact that the application of intelligent systems can simplify the decision-making process in financial markets, which is a substantial advantage. Furthermore, by comparing the predicted outcome errors between the models, the LSTM presents the lowest error with a higher computational time in the training phase. The LSTM was able to accurately forecast the closing price of the day as well as the price for the following one and two days in advance. In conclusion, the empirical results demonstrated that these models could accurately predict financial data for trading purposes and that the application of intelligent systems, such as the LSTM network, represents a promising advancement in financial technology.

Stabilizing the Bulk-Phase and Solid Electrolyte Interphase of Silicon Microparticle Anode by Constructing Gradient-Hierarchically Ordered Conductive Networks.

The poor bulk-phase and interphase stability, attributable to adverse internal stress, impede the cycling performance of silicon microparticles (µSi) anodes and the commercial application for high-energy-density lithium-ion batteries. In this work, a groundbreaking gradient-hierarchically ordered conductive (GHOC) network structure, ingeniously engineered to enhance the stability of both bulk-phase and the solid electrolyte interphase (SEI) configurations of µSi, is proposed. Within the GHOC network architecture, two-dimensional (2D) transition metal carbides (Ti3C2Tx) act as a conductive "brick", establishing a highly conductive inner layer on µSi, while the porous outer layer, composed of one-dimensional (1D) Tempo-oxidized cellulose nanofibers (TCNF) and polyacrylic acid (PAA) macromolecule, functions akin to structural "rebar" and "concrete", effectively preserves the tightly interconnected conductive framework through multiple bonding mechanisms, including covalent and hydrogen bonds. Additionally, Ti3C2Tx enhances the development of a LiF-enriched SEI. Consequently, the µSi-MTCNF-PAA anode presents a high discharge capacity of 1413.7 mAh g-1 even after 500 cycles at 1.0 C. Moreover, a full cell, integrating LiNi0.8Mn0.1Co0.1O2 with µSi-MTCNF-PAA, exhibits a capacity retention rate of 92.0% following 50 cycles. This GHOC network structure can offer an efficacious pathway for stabilizing both the bulk-phase and interphase structure of anode materials with high volumetric strain.

A Novel Multi-Scale Feature Map Fusion for Oil Spill Detection of SAR Remote Sensing

The efficient and timely identification of oil spill areas is crucial for ocean environmental protection. Synthetic aperture radar (SAR) is widely used in oil spill detection due to its all-weather monitoring capability. Meanwhile, existing deep learning-based oil spill detection methods mainly rely on the classical U-Net framework and have achieved impressive results. However, SAR images exhibit high noise, blurry boundaries, and irregular shapes of target areas, as well as speckles and shadows, which lead to the loss of performance in existing algorithms. In this paper, we propose a novel network architecture to achieve more precise segmentation of oil spill areas by reintroducing rich semantic contextual information before obtaining the final segmentation mask. Specifically, the proposed architecture can re-fuse feature maps from different levels at the decoder end. We design a multi-convolutional layer (MCL) module to extract basic feature information from SAR images, and a feature extraction module (FEM) module further extracts and fuses feature maps generated by the U-Net decoder at different levels. Through these operations, the network can learn rich global and local contextual information, enable sufficient interaction of feature information at different stages, enhance the model’s contextual awareness, and improve its ability to recognize complex textures and blurry boundaries, thereby enhancing the segmentation accuracy of SAR images. Compared to many U-Net based segmentation networks, our method shows promising results and achieves state-of-the-art performance on multiple evaluation metrics.